Finite / Countable State Space Stochastic Processes : Point Processes: 
Characterization of Associated Auto-Correlation Functions: 


Garimella Rama Murthy, 
Associate Professor, 
MT—Hyderabad, Gachibowli, 
HYDERABAD-32, AP, INDIA 


ABSTRACT 


In this research paper, the relationship between finite / countable state space 
stochastic processes and point processes is explored. Utilizing the known relationship 
between Poisson processes and continuous time Markov chains, finite / countable state 
space random processes are related to continuous time Markov Chains. Based on the 
known results for binary random processes, characterization of auto-correlation 
function of finite state space random processes is explored. An important 
characterization of corner positive definite matrices is provided. 


1. Introduction: 

Most natural as well as artificial dynamic phenomena are endowed with 
a stochastic rather than deterministic description. Thus, such a phenomena are 
mathematically modeled using a stochastic process. Among the class of stochastic 
processes utilized in stochastic modeling, Markovian processes assume special importance. 
The main reason is the existence of equilibrium probabilistic behaviour which can be 
computed efficiently. Thus, many equilibrium performance measures are efficiently 
determined. The phenomena modeled by Markovian stochastic processes satisfy the 
property that conditional on present and past, the future depends only on the present. 
This modeling assumption is satisfied in many stochastic models of natural and artificial 
phenomena. Phenomena occuring in higher dimensions are modeled using Markov 
Random Fields. 

In some applications, events occur on the real line at random times. 
Such phenomena are modeled using stochastic processes called point processes. Point 
processes are utilized in stochastic models arising in queueing, inventory and 
communication theories. Among point processes, Poisson process is widely utilized 
in tractable stochastic models. In the case of Poisson process, the interarrival time 
distributions are exponential. The exponential distribution is the only such distribution 
which satisfies a property called “memorylessness” [Ross]. 

Researchers attempted and succeeded 
in relating Poisson process and continuous time Markov chains. Based on this 
connection, the author became interested in exploring the relationship between arbitrary 
finite / countable state space stochastic processes and Point processes. The author hopes 
that detailed investigation of this relationship enables the analysis of “structured finite / 
countable state space stochastic processes” using the theory of continuous time Markov 
chains. 

This research paper is organized as follows. In Section 2, relationship between 
finite / countable state space stochastic processes and point processes is explored. In 
Section 3, countable state space random processes and countable state space Markov 
chains are related. In Section 4, characterization of autocorrelation function of finite state 
space random processes is explored. The research paper concludes in Section 5. 


2. Relationship between Finite /Countable State Space Stochastic Processes and 


Point Processes: 
Let the state space of a finite state space stochastic process be 


{ 1,2, ....,N }. Considering the random variables representing the successive sojourn 
times in state “j? ( 1<j<N } be 
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These random variables can be treated as the inter-arrival times of the “j’th point 
process. Thus, we have the following claim. 


Claim 1: An arbitrary finite state space random process can be treated as the 
superposition of finitely many competing point processes. 


The above claim is a generalization of the following claim. 


Claim 2: Any finite state space Continuous Time Markov Chain (CTMC) 
can be treated as the superposition of finitely many competing Poisson 
processes [Cin]. 
The above claims have natural generalization to 
countable state space stochastic processes ( under some regularity conditions). 


Application of the above Claims : 

In a large packet network, at routers / switches, 
one observes the superposition of packet streams. Thus, we are naturally led to 
the analytical investigation of superposition of point processes ( modeling the 
packet streams ). Also, there are several other applications where it is necessary 
to characterize the superposition of finitely /countably many point processes. 


Countable State Space Random Processes : Countable State Space Markov Chains: 


From theoretical as well as practical considerations, one is interested in 
superposition of countably many point processes. The following claim is relevant in 
such a discussion and follows from a generalization of the ideas in Section 2. 


Claim 3: A countable state space stochastic process can be treated as the 
superposition of countably many point processes. 


Now, we are interested when a countable state space stochastic process 
constitutes a Continuous Time Markov Chain (CTMC) or more generally when it 
can be approximated by a CTMC. The following claim provides one possible 
answer in the proper direction and it is well understood in the point process 


literature. For mathematically formal statement, one can refer to any standard textbook 
on point processes. 


Claim 4: The superposition of countably many, “uniformly sparse” point 


processes constitutes a Poisson process. 


The condition of “uniform sparseness” of point processes is well 
understood, formalized and documented in the literature. 


The claim 4 is carefully interpreted to arrive at the relationship between 
countable state space stochastic processes and countable state space Markov 


chains. 


e Consider an infinite state space stochastic process as the superposition of 
countably many competing point processes. When these competing point 
processes are “uniformly sparse”, let the stochastic process be of 
“SPECIAL TYPE”. 


Theorem 1: Any “special type” stochastic process constitutes a countable state 
space “Uniformized” Continuous Time Markov Chain. 


Remark: It is well known that the Central Limit Theorem has theoretical 
as well as practical significance. We expect the above theorem to 
be of utility in various applications including characterization of 
packet streams in point-to-point networks. 


4. Characterization of Auto-Correlation Function of Finite State Space Random 


Processes: 
It is well known from Bochner-Herglotz’s Theorem that the 


autocorrelation function of a stationary random process needs to be a positive 
definite function. But researchers such as Mc-Millan became interested in 
characterizing the autocorrelation function ( equals the auto-covariance function when 
the mean of the stationary random process is zero ) of stationary random process 
which assumes the values { +1, -1 } ( with equal probability so that the mean is zero ). 


P (n) = E {XmXmen } 


Let the class of functions be denoted by U.In fact Mc-Millan’s stated the following 
Theorem: 


Theorem 2: (B. Mc Millan ): 


peU & (if and only if ) 
p(0)= 1 and 
nai Emai p(m a n) Amn 20 
for every N, and all corner-positive matrices {Amn}, m,n = 1,2,...,N, where 
a matrix {Amn} is corner-positive if 
nA nsi Amn Em En 2 0 
for every sequence (€),...,€y ) with €; = F1, i= 1,2,...,N. 
The above characterization of auto-correlation function of a unit random 
process can be reformulated as follows: 
p(0)= 1 and 
Trace ( R X ) = Trace ( X R ) => 0 (linear form in the components of X ) 
where R is NxN autocorrelation matrix (corresponding to various lags ) 
for every N, and all corner-positive matrices {Xm}, m,n = 1,2,...,N, where 
a matrix {Xmn} is corner-positive if 
eX €=0 (i.e.quadratic form associated with X ) 
for every vector e" = (€,..,€y ) lying on the symmetric binary hypercube. 


e Thus, we can denote the class U of unit autocorrelation functions as 
“comer-positive definite” functions. The above characterization problem naturally deals 
with determining the properties of “corner positive (definite ) ” matrices. 

As a natural generalization, we introduce the following definitions related 
to positive definiteness of matrices on interesting subsets of R"/C”. 


(A) V-positive Definite Matrix: If a matrix is positive definite at all points of V i.e. 
XTAX>O for all XeV, 
then, the matrix A is said to be a V positive Definite matrix. 
Note: The set V can be any finite /countable / uncountable set of R” or C”. 
For instance, if V happens to be bounded, symmetric lattice, we call A to 
be “Bounded Lattice Positive Definite” matrix. Also V can be the entire 
lattice in R”. 


Note: As a natural generalization, we can define V-positive Definite functions 
(in the spirit of Bochner—Herglotz’s Theorem ). These could be called 
Corner Positive Definite functions, Lattice positive definite functions etc. 


Now, we start with a characterization of Corner Positive Definite 
matrices. It should be kept in mind that the basic approach works for bounded- 
lattice positive definite matrices. 


Theorem 3: A matrix C (defined on the field of real or complex numbers ) is 
Corner Positive Definite (CPD) 


(ie. X™C X 20 for all vectors X lying on the symmetric binary hypercube ) 
if and only if 
XTE X > Trace (C) for all vectors X such that 
X = —sign (E X ) where sign (.)is the signum function 
and Eis the symmetric matrix with zero diagonal elements obtained from D, 
where D = - (C+C? ). 


Proof: The proof follows based on non-linear optimization techniques ( maximum 
principle or Kuhn-Tucker conditions ) and is avoided for brevity. It is related to 
the Theorem proved in [Rama]. Q.E.D. 


Remark: Hopfield neural network leads to the optimization ofa quadratic form on 
the symmetric binary hypercube, H. It is proved that the local/global optimum 
occur at the stable states i.e. 


Xe H such that X = Sign( EX). 


The author alongwith his student Praveen generalized the Theorem to the case 
where C is a complex valued matrix [Rama]. 


e Jt should be noted that in the above Theorem, we are interested in 
determining the local minimum of quadratic form evaluated on the 
symmetric binary hypercube. Let the local minimum vectors on the 
symmetric binary hypercube be called the “anti-stable states”. The ith 


component of a candidate optimal solution (i.e. an anti-stable state ) can be 
computed by the following successive approximation procedure: 

As in the case of serial mode of operation of Hopfield neural network, 
the proof of convergence of the above successive approximation scheme 
can easily be provided. Details are avoided for brevity. Also the author 
formulated and solved the problem of minimizing the quadratic form 
associated with a Hermitian matrix on the hypercube. 


e In view of the above discussion, there is no loss of generality in assuming 
X to be a symmetric zero diagonal matrix in Theorem 3. 
e From basic linear algebra, we infer the following: 
Trace (RX ) = Trace (XR) with Trace (X)=0 and X,R are both 
symmetric matrices. Also, it is well known that both the matrices 
XR, RX have the same characterisitic polynomial. 


Now, we consider the generalization of Mc-Millan’s Theorem 
to the characterization of autocorrelation function ( auto-covariance function 
of astationary zero mean process ) of stationary finite state space random 
processes. 

e Let the class of such autocorrelation functions of a finite state space 
random process, Xn 
(X,= +], Je{t1,+2,....,+M } 
with the condition that 
1 
P{X, = +j }=P{X, = -j} = 2M 
be denoted by V. 


Theorem 4: peV © (if and only if ) 
M+1)(2M+1 


a ei p(m =, n) Bmn 20 


for every M, and all bounded, symmetric lattice positive definite matrices 
{Bmn} m,n = 1,2,...,M, where a matrix {Bmn} is bounded symmetric lattice positive 
definite if 
A Jei Bmn Em En 2 0 
for every sequence ( €41, ...,E€4 ) lying on the bounded symmetric lattice. 


Proof: Necessity and sufficiency follow from a generalization of the argument 
used to prove the Mc Millan’s Theorem. It is avoided for brevity Q.E.D. 


Note: The generalization of above Theorem for countable state space random 
processes follows by doing the optimization on the lattice. It can be 
compared with the Bochner-Herglotz’s Theorem. 


Remark: Bruck et.al considered the problem of maximization of higher degree 
forms (than quadratic forms ) on the bounded lattice [Rama]. These results 
naturally suggest the minimization of quadratic forms over the bounded symmetric 


lattice. Thus, the characterization of bounded symmetric lattice positive definite 
matrices follows from the results in constrained optimization. 


5. Conclusions: 

In this research paper, point processes are related to finite / countable 
state space stochastic processes. Using the well known relationship between, Poisson 
processes and finite state space continuous time Markov chains, structured countable state 
space random processes are related to countable state space continuous time Markov 
chain. Also, continuous time Markov chain model of channels in point-to-point networks 
is proposed. 


References: 
[Cin] E. Cinlar, “Stochastic Processes”, Academic Press 
[Ross] S.M.Ross, “Introduction to Probability Models, “ Academic Press, 2004 


[Rama] G. Rama Murthy,’Multi-Dimensional Neural Networks : Unified Theory,’ New 
Age International Publishers, New Delhi 


